Deep Orthogonal Fusion: Multimodal Prognostic Biomarker Discovery Integrating Radiology, Pathology, Genomic, and Clinical Data

Clinical decision-making in oncology involves multimodal data such as radiology scans, molecular profiling, histopathology slides, and clinical factors. Despite the importance of these modalities individually, no deep learning framework to date has combined them all predict patient prognosis. Here, we overall survival (OS) glioma patients from diverse with a Deep Orthogonal Fusion (DOF) model. The model learns combine information multiparametric MRI exams, biopsy-based (such H&E slide images and/or DNA sequencing), variables into comprehensive risk score. Prognostic embeddings each modality are learned via attention-gated tensor fusion. To maximize gleaned modality, introduce orthogonalization (MMO) loss term that increases performance by incentivizing constituent be more complementary. DOF predicts OS median C-index 0.788 ± 0.067, significantly outperforming (p = 0.023) best performing unimodal 0.718 0.064. prognostic stratifies within subsets, adding further granularity grading subtyping.